Skip to content

Actually wire the pluggable expression convertor#7730

Merged
AdamGS merged 1 commit intodevelopfrom
adamg/pluggable-expr-convertor-fix
Apr 30, 2026
Merged

Actually wire the pluggable expression convertor#7730
AdamGS merged 1 commit intodevelopfrom
adamg/pluggable-expr-convertor-fix

Conversation

@AdamGS
Copy link
Copy Markdown
Contributor

@AdamGS AdamGS commented Apr 30, 2026 •
edited
Loading

Summary

Turns out I didn't wire the expression convertor extension point correctly. This PR both fixes that AND adds a test to make sure this behavior is maintained.

@AdamGS
Actually wire the pluggable expression convertor
fd87612 
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@AdamGS AdamGS added changelog/fix A bug fix ext/datafusion Relates to the DataFusion integration labels Apr 30, 2026
@AdamGS AdamGS requested a review from a10y April 30, 2026 14:37
AdamGS
AdamGS commented Apr 30, 2026
View reviewed changes
Comment thread vortex-datafusion/src/persistent/source.rs

impl FileSource for VortexSource {
fn create_file_opener(
fn create_vortex_opener(
Copy link
Copy Markdown
Contributor Author

@AdamGS AdamGS Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

introduce new private function because currently FileSource can't be downcasted, so we want to return a real type in tests.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Apr 30, 2026

Merging this PR will degrade performance by 10.6%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
❌ 1 regressed benchmark
✅ 1193 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ WallTime runend[10M_i32_runlen_10] 195.1 µs 160.4 µs +21.63%
⚡ WallTime runend[10M_i32_runlen_1000] 135.2 µs 97.2 µs +39.13%
⚡ WallTime dynamic_dispatch_u32[10M] 161.8 µs 108.5 µs +49.06%
⚡ WallTime dynamic_dispatch_u32[10M] 169 µs 115.7 µs +46.02%
❌ Simulation bitwise_not_vortex_buffer_mut[128] 246.1 ns 275.3 ns -10.6%

Comparing adamg/pluggable-expr-convertor-fix (fd87612) with develop (c73dbb2)

Open in CodSpeed

@AdamGS AdamGS added changelog/fix A bug fix and removed changelog/fix A bug fix labels Apr 30, 2026
@AdamGS AdamGS merged commit 40112f4 into develop Apr 30, 2026
84 of 90 checks passed
@AdamGS AdamGS deleted the adamg/pluggable-expr-convertor-fix branch April 30, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@robert3005 robert3005 robert3005 approved these changes

@a10y a10y Awaiting requested review from a10y

Assignees

No one assigned

Labels

changelog/fix A bug fix ext/datafusion Relates to the DataFusion integration

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Custom ExpressionConvertor is not propagated from VortexSource to VortexOpener

2 participants

@AdamGS @robert3005